Effective software testing is crucial to achieving a competitive software product. Testing may be part of any stage of the product development, including design, unit testing, system testing or the like. Testing may be planned and implemented by generating and executing a test model.
A combinatorial model, also referred to as Cartesian-product model, is a set of attributes, values or value ranges for the attributes (also referred to as domains), and restrictions on value combinations that may not appear together. Such a model spans a space of valid tests for a tested design, unit, system or any other entity: each combination of values to the attributes that does not violate any restriction corresponds to a valid test. Each combination of attributes or a subset of attributes, which is excluded by one or more restrictions, is referred to as an illegal combination.
For example, a model for testing a computer may comprise a CPU attribute with possible values consisting of the available CPUs, an operating system (OS) attribute with possible values consisting of the available Oss, and restrictions that rule out impossible of CPU and OS combinations, such as a low performance CPU with modern high-resource-consuming OS.
Combinatorial models have a variety of usages. One such usage is functional coverage analysis of a System Under Test (SUT). Another usage is in Combinatorial Test Design (CTD). CTD is a test planning technique that selects a small subset of the valid test space that covers a predefined coverage goal. The coverage goal may define an interaction level of attributes that are to be covered, such as every n-wise combinations of values.
Preparing a combinatorial model may be a difficult task for a user, such as a verification engineer, a QA staff member, or the like. In particular, correctly capturing the restrictions over the model may be an important yet work-intensive and non-trivial task. Under-restricting the model may yield tests that cannot be executed and may cause coverage gaps if such tests are skipped or manually modified. Over-restricting the model also yields coverage gaps in the areas that are wrongly restricted.